Representative image generation

ABSTRACT

Representative image generation technology is provided. A representative image generation method according to embodiments of the present invention includes automatically generating a representative image for a relevant content by recognizing an object in the content, such as a webtoon, an illustration, multiple images related to a particular product, or a user&#39;s photograph album.

CROSS-REFERENCE TO RELATED APPLICATIONS

This is a continuation of International Application No.PCT/KR2018/008402, filed Jul. 25, 2018, which claims benefit of KoreanPatent Application No. 10-2017-0124444, filed Sep. 26, 2017.

BACKGROUND OF THE INVENTION Field of Invention

One or more example embodiments of the following description relate torepresentative image generation technology, and more particularly, to arepresentative image generation method that may recognize an object incontent, such as a webtoon, an illustration, a plurality of imagesrelated to a specific product, a photo album of a user, and the like,and may automatically generate a representative image for thecorresponding content. The present invention also relates to a computerapparatus for performing the representative image generation method, acomputer program stored in a non-transitory computer-readable recordmedium to perform the representative image generation method on acomputer in conjunction with the computer, and a non-transitorycomputer-readable record medium for storing a computer program forenabling a computer to perform the representative image generation.

Description of Related Art

The term “webtoon” is a combination of web indicating the Internet andcartoon meaning comics. Such webtoons are being serviced in variousforms, such as comics in a vertical image file format displayed on awebsite, and comics in which a scene of a cut unit is displayed on ascreen.

Also, various technologies are present to service such webtoons tousers. For example, a cartoon providing system, a cartoon providingapparatus, and a cartoon providing method are described in Korean PatentLaid-Open Publication No. 10-2014-0040875. Disclosed is technology forproviding a plurality of cut images constituting a cartoon.

Here, many technologies for servicing webtoons to users generate and userepresentative images of webtoons to introduce the webtoons to theusers. For example, webtoons may be displayed to users through websitesor mobile pages, based on various conditions, for example, a publisheddate, a completion status, rankings, order of titles, and the like.Here, each of the webtoons may be displayed with information used forthe users to identify a corresponding webtoon such as a representativeimage of the webtoon and a title of the webtoon, and/or information usedto draw interest of the users on the corresponding webtoon. Here, therepresentative image may include an image that represents a single wholewebtoon having a series of episodes and an image that represents eachepisode of the corresponding webtoon.

A service provider that provides webtoons to users needs to generate arepresentative image for each of the webtoons provided through aservice. Here, as a number of webtoons to be serviced increases, costused to generate a representative image also increases. For example, ifnew episodes of tens of webtoons are registered and serviced a day, alarge amount of time and efforts are required for the service providerto verify all of the contents of the respective episodes of each of tensof webtoons and to generate representative images suitable for therespective episodes. Also, since information about the plurality ofwebtoons or the plurality of episodes is displayed on a single screen, asize of a representative image is limited, which makes it difficult touse a cut image of a webtoon as a representative image. That is, togenerate a representative image, additional cost is required to select asingle image from among a large number of images and to select andextract a main portion from the selected image, instead of simplyselecting a single image from among the plurality of images.

Also, users expect a representative image for each of various featuresof the webtoons. However, to meet such expectation, a larger amount ofresources are required. For example, webtoons may be displayed for usersin a form of a list in which the webtoons are classified based onvarious conditions, for example, rankings, genres, and the like.However, as described above, a large amount of resources are required tosimply generate a representative image for each webtoon and for eachepisode of each webtoon. Therefore, it is difficult to generate and usevarious types of representative images based on various ratios and/orsizes by considering features of each list and/or each user interface.

Also, there is a need to select a representative image of specificcontent, such as an illustration, a photo album of a user, and aplurality of images provided for a specific product in addition to theaforementioned webtoon.

BRIEF SUMMARY OF THE INVENTION

One or more example embodiments provide a representative imagegeneration method that may recognize an object in content, such as awebtoon, an illustration, a plurality of images related to a specificproduct, a photo album of a user, and the like, and may automaticallygenerate a representative image for the corresponding content, acomputer apparatus for performing the representative image generationmethod, a computer program stored in a non-transitory computer-readablerecord medium to perform the representative image generation method on acomputer in conjunction with the computer, and the non-transitorycomputer-readable record medium.

One or more example embodiments provide a representative imagegeneration method that may provide a tool capable of automaticallygenerating and managing a representative image using an image matchingmodel, a computer apparatus for performing the representative imagegeneration method, a computer program stored in a non-transitorycomputer-readable record medium to perform the representative imagegeneration method on a computer in conjunction with the computer, andthe non-transitory computer-readable record medium.

According to an aspect of at least one example embodiment, there isprovided a representative image generation method including recognizingan object in at least one image included in content; generating arecognition result image by extracting an area including the recognizedobject from the at least one image; and generating a representativeimage related to the content based on the generated recognition resultimage.

According to an aspect of at least one example embodiment, there isprovided a computer program stored in a non-transitory computer-readablerecord medium to perform the representative image generation method on acomputer in conjunction with the computer.

According to an aspect of at least one example embodiment, there isprovided a non-transitory computer-readable record medium storing aprogram to perform the representative image generation method on acomputer.

According to an aspect of at least one example embodiment, there isprovided a computer apparatus including at least one processorconfigured to execute a computer-readable instruction. The at least oneprocessor is configured to recognize an object in at least one imageincluded in content, generate a recognition result image by extractingan area including the recognized object from the at least one image, andgenerate a representative image related to the content based on thegenerated recognition result image.

According to some example embodiments, it is possible to recognize anobject in content, such as a webtoon, an illustration, a plurality ofimages related to a specific product, a photo album of a user, and thelike, and to automatically generate a representative image for thecorresponding content.

According to some example embodiments, it is possible to provide a toolcapable of automatically generating and managing a representative imageusing an image matching model.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating an example of a network environmentaccording to an example embodiment.

FIG. 2 is a block diagram illustrating a configuration of an electronicdevice and a server according to an example embodiment.

FIG. 3 illustrates an example of a representative image generationprocess according to an example embodiment.

FIG. 4 illustrates an example of a process of outputting a final imagein a target image according to an example embodiment.

FIGS. 5 to 8 illustrate examples of a tool for generating and managing arepresentative image according to an example embodiment.

FIG. 9 is a flowchart illustrating an example of a representative imagegeneration method according to an example embodiment.

DETAILED DESCRIPTION OF THE INVENTION

Hereinafter, example embodiments will be described with reference to theaccompanying drawings.

A representative image generation method according to exampleembodiments may be performed through a computer apparatus or a serverthat is described below. Here, a computer program according to exampleembodiments may be installed and executed on the computer apparatus, andthe computer apparatus may perform the representative image generationmethod according to the example embodiments under control of theexecuted computer program. The computer program may be stored in anon-transitory computer-readable record medium to perform therepresentative image generation method on a computer in conjunction withthe computer apparatus.

FIG. 1 is a diagram illustrating an example of a network environmentaccording to at least one example embodiment. Referring to FIG. 1, thenetwork environment includes a plurality of electronic devices 110, 120,130, and 140, a plurality of servers 150 and 160, and a network 170.FIG. 1 is provided as an example only and thus, a number of electronicdevices or a number of servers are not limited thereto.

Each of the plurality of electronic devices 110, 120, 130, and 140 maybe a fixed terminal or a mobile terminal configured as a computerapparatus. For example, the plurality of electronic devices 110, 120,130, and 140 may be a smartphone, a mobile phone, a navigation, acomputer, a laptop computer, a digital broadcasting terminal, a personaldigital assistant (PDA), a portable multimedia player (PMP), and atablet personal computer (PC). For example, although FIG. 1 illustratesa shape of a smartphone as an example of the electronic device (1) 110,the electronic device (1) 110 may indicate one of various physicalcomputer apparatuses capable of communicating with other electronicdevices 120, 130, and 140, and/or the servers 150 and 160 over thenetwork 170 in a wireless communication manner or in a wiredcommunication manner.

The communication scheme is not particularly limited and may include acommunication method using a near field communication between devices aswell as a communication method using a communication network, forexample, a mobile communication network, the wired Internet, thewireless Internet, a broadcasting network, etc., which may be includedin the network 170. For example, the network 170 may include at leastone of network topologies that include, for example, a personal areanetwork (PAN), a local area network (LAN), a campus area network (CAN),a metropolitan area network (MAN), a wide area network (WAN), and theInternet. Also, the network 170 may include at least one of networktopologies that include a bus network, a star network, a ring network, amesh network, a star-bus network, a tree or hierarchical network, andthe like. However, it is provided as an example only and the exampleembodiments are not limited thereto.

Each of the servers 150 and 160 may be configured as a computerapparatus or a plurality of computer apparatuses that providesinstructions, codes, files, contents, services, and the like, throughcommunication with the plurality of electronic devices 110, 120, 130,and 140 over the network 170. For example, the server 150 may be asystem that provides a first service associated with the plurality ofelectronic devices 110, 120, 130, and 140 connected over the network170. The server 160 may be a system that provides a second serviceassociated with the plurality of electronic devices 110, 120, 130, and140 connected over the network 170. In detail, for example, the server150 may provide a webtoon service as the first service. In this case,the server 150 may generate and use a representative image of each ofwebtoons to be displayed through a webtoon service using therepresentative image generation method according to example embodiments.

FIG. 2 is a block diagram illustrating an example of an electronicdevice and a server according to at least one example embodiment. FIG. 2illustrates a configuration of the electronic device (1) 110 as anexample for a single electronic device and illustrates a configurationof the server 150 as an example for a single server. The same or similarcomponents may be applicable to other electronic devices 120, 130, and140, or the server 160.

Referring to FIG. 2, the electronic device (1) 110 may include a memory211, a processor 212, a communication module 213, and an input/output(I/O) interface 214, and the server 150 may include a memory 221, aprocessor 222, a communication module 223, and an I/O interface 224. Thememory 211, 221 may include a permanent mass storage device, such asrandom access memory (RAM), a read only memory (ROM), and a disk drive,as a non-transitory computer-readable record medium. The permanent massstorage device such as ROM and disk drive may be included in theelectronic device (1) 110 or the server 150 as a permanent storagedevice separate from the memory 211, 221. Also, an OS or at least oneprogram code, for example, a code for a browser installed and executedon the electronic device (1) 110 or an application installed andexecuted on the electronic device (1) 110 to provide a specific service,may be stored in the memory 211, 221. Such software components may beloaded from another non-transitory computer-readable record mediumseparate from the memory 211, 221. The other non-transitorycomputer-readable record medium may include a non-transitorycomputer-readable record medium, for example, a floppy drive, a disk, atape, a DVD/CD-ROM drive, a memory card, etc. According to other exampleembodiments, software components may be loaded to the memory 211, 221through the communication module 213, 223, instead of the non-transitorycomputer-readable record medium. For example, at least one program maybe loaded to the memory 211, 221 based on a computer program, forexample, the application, installed by files provided over the network170 from developers or a file distribution system, for example, theserver 160, which provides an installation file of the application.

The processor 212, 222 may be configured to process computer-readableinstructions of a computer program by performing basic arithmeticoperations, logic operations, and I/O operations. The computer-readableinstructions may be provided from the memory 211, 221 or thecommunication module 213, 223 to the processor 212, 222. For example,the processor 212, 222 may be configured to execute receivedinstructions in response to the program code stored in the storagedevice, such as the memory 211, 221.

The communication module 213, 223 may provide a function forcommunication between the electronic device (1) 110 and the server 150over the network 170 and may provide a function for communication withanother electronic device, for example, the electronic device (2) 120 oranother server, for example, the server 160. For example, the processor212 of the electronic device (1) 110 may transfer a request createdbased on a program code stored in the storage device, such as the memory211, to the server 150 over the network 170 under control of thecommunication module 213. Inversely, a control signal, an instruction,content, a file, etc., provided under control of the processor 222 ofthe server 150 may be received at the electronic device (1) 110 throughthe communication module 213 of the electronic device (1) 110 by goingthrough the communication module 223 and the network 170. For example, acontrol signal, an instruction, content, a file, etc., of the server 150received through the communication module 213 may be transferred to theprocessor 212 or the memory 211, and content, a file, etc., may bestored in a record medium, for example, the permanent storage device,further includable in the electronic device (1) 110.

The I/O interface 214 may be a device used for interface with an I/Oapparatus 215. For example, an input device of the I/O apparatus 215 mayinclude a device, such as a keyboard and a mouse, and an output deviceof the I/O apparatus 215 may include a device, such as a display and aspeaker. As another example, the I/O interface 214 may be a device forinterface with an apparatus in which an input function and an outputfunction are integrated into a single function, such as a touchscreen.The I/O apparatus 215 may be configured as a single device with theelectronic device (1) 110. Also, the I/O interface 224 of the server 150may be a device for interface with an apparatus (not shown) for input oroutput that may be connected to the server 150 or included in the server150. In detail, when processing instructions of the computer programloaded to the memory 211, the processor 212 of the electronic device (1)110 may display a service screen configured using data provided from theserver 150 or the electronic device (2) 120, or may display content on adisplay through the I/O interface 214.

According to other example embodiments, the electronic device (1) 110and the server 150 may include a greater number of components than anumber of components shown in FIG. 2. However, there is no need toclearly illustrate many components according to the related art. Forexample, the electronic device (1) 110 may include at least a portion ofthe I/O apparatus 215, or may further include other components, forexample, a transceiver, a global positioning system (GPS) module, acamera, a variety of sensors, a database (DB), and the like. In detail,if the electronic device (1) 110 is a smartphone, the electronic device(1) 110 may be configured to further include a variety of components,for example, an accelerometer sensor, a gyro sensor, a camera module,various physical buttons, a button using a touch panel, an I/O port, avibrator for vibration, etc., which are generally included in thesmartphone.

FIG. 3 illustrates an example of a representative image generationprocess according to an example embodiment. An example of generating arepresentative image for a single webtoon as content is described withreference to FIG. 3. In FIG. 3, processes 310 to 370 may be performed bythe server 150. And, an object recognizer 321, a character facerecognizer 322, a speech balloon recognizer 323, and a text recognizer324 may be functional expressions of the processor 222 of the server150.

A target content input process 310 may be an example of a process ofreceiving one or more target images included in a webtoon. For example,the target content input process 310 may be a process of loading the oneor more target images to the memory 221 of the server 150. The targetimages may be stored in a local storage of the server 150 in advance, ormay be received from an external device. Here, a subsequent objectrecognition process 320 may be performed with respect to each of thereceived target images. For example, in the webtoon, a single episodemay be configured as a single target image and may be configured as aplurality of target images classified for each cut. In this case, theobject recognition process 320 may be performed with respect to each ofthe target images.

The object recognition process 320 may be an example of a process ofrecognizing an object in the received target image. The objectrecognition process 320 may be performed by an object recognizer 321.The object recognizer 321 may include, for example, a character facerecognizer 322, a speech balloon recognizer 323, and a text recognizer324.

The character face recognizer 322 may recognize a face of a character asan object in a target image, the speech balloon recognizer 323 mayrecognize a speech balloon as an object in the target image, and thetext recognizer 324 may recognize a text as an object in the targetimage. Although examples of recognizing a character face, a speechballoon, and a text are described as a pattern of an object to berecognized, it is provided as an example only. The pattern of the objectmay variously use a shape of a person or a shape of an animal and/or ashape of a vehicle, such as a car or an airplane.

Pattern-by-pattern learning data 325 may be used to recognize each ofpatterns. The pattern-by-pattern learning data 325 may be used to trainthe object recognizer 321 and/or may be used as reference data of anobject that is desired to be extracted in an image. For example, thepattern-by-pattern learning data 325 may be generated using an existingcontent and target images extracted by a person from the existingcontent.

As an example of learning, images including faces of various charactersmay be input as learning data to the character face recognizer 322,images including various speech balloons may be input to the speechballoon recognizer 323, and texts may be input to the text recognizer324 as learning data. Here, each of the character face recognizer 322,the speech balloon recognizer 323, and the text recognizer 324 may learna function for recognizing each required pattern through machinelearning using the input learning data. A method of training recognizersthrough machine learning and using the trained recognizers may be easilyunderstood by those skilled in the art through known arts in associationwith machine learning.

As an example of reference data, an image including a character facethat is a standard of an object to be recognized may be input asreference data to the character face recognizer 322. In this case, thecharacter face recognizer 322 may recognize a character face in a targetimage based on the character face of the image input as learning data.Similar thereto, an image including a speech balloon that is a standardof an object desired to be recognized may be input as reference data tothe speech balloon recognizer 323 and a text that is a standard of anobject to be recognized may be input as reference data to the textrecognizer 324. In detail, if a text “car” is input as reference data,the text recognizer 324 may search a target image to determine whetherthe text “car” is included.

As described above, the object recognition process 320 may be performedwith respect to each of the target images input through the targetcontent input process 310. Here, if a plurality of target images ispresent, a target image in which an object is recognized may be presentand a target image in which an object is not recognized may be present.Also, a plurality of objects may be recognized in a single target image.Here, the object recognizer 321 may extract an area including an objectrecognized in a target image and may generate a recognition result imagefor each recognized object. For example, the character face recognizer322 may extract an area including a character face recognized in atarget image and may generate a recognition result image with a sizepreset for the extracted area.

An image size selection process 330 may be an example of a process ofdetermining a size, for example, 120×120 pixels or 96×96 pixels, of arepresentative image to be extracted in the recognition result image.

A size-by-size object position guide selection process 340 may be anexample of a process of determining a position of an object in arepresentative image. For example, FIG. 3 illustrates an example 341 foraligning and then cropping a position of an object in a representativeimage to one of the left, right, and center of the representative image,an example 342 for aligning and then cropping the position of the objectto one of the top, middle, and bottom, and an example 343 for enlargingor reducing and then cropping the position of the object. Depending onexample embodiments, the position of the object may be determined basedon further various guides to be aligned to upper left, lower right, ormiddle left and then enlarged, or to be aligned to upper right and thenreduced.

A primary image output process 350 may be an example of a process ofoutputting the representative image extracted in the recognition resultimage, and an operator inspection and edition process 360 may be anexample of a process of inspecting, by an operator, and/or editing theoutput representative image. Also, a final image output process 370 maybe an example of a process of outputting the representative imageinspected and/or edited by the operator. A final image may be displayedthrough a site for providing a webtoon to users. Here, the primary imageoutput process 350 and the operator inspection and edition process 360may be omitted depending on example embodiments. In this case, theprimary image output process 350 may be a process of displaying therepresentative image extracted from the recognition result image throughthe site for providing the webtoon to users, which is similar to thefinal image output process 370. For example, a predetermined period maybe required to train the server 150 to automatically process the objectrecognition process 320, the image size selection process, 330, and thesize-by-size object position guide selection process 340. That is, alearning period may be required such that the server 150 mayautomatically generate a representative image suitable for input targetcontent. Before the learning period is completed, the operator needs todirectly inspect and/or edit the generated representative image. Also,data collected according to inspection and/or edition of the operatormay be used as learning data for the server 150 to generate therepresentative image during the learning period.

FIG. 4 illustrates an example of a process of outputting a final imagein a target image according to an example embodiment. FIG. 4 illustratesan example of a target image 410 input through the target content inputprocess 310 of FIG. 3 and also illustrates an example of recognizing, bythe character face recognizer 322, an object in the target image 410using an image 420 input as reference data in the object recognitionprocess 320. Here, a circle 430 indicated with a solid line in thetarget image 410 of FIG. 4 represents an example of the objectrecognized in the target image 410.

Also, FIG. 4 illustrates an example of a primary image 440 extractedthrough the image size selection process 330 and the size-by-size objectposition guide selection process 340. Here, the primary image 440 has asize of “120×120 pixels” and the object is positioned at an upper centerof the primary image 440.

Also, FIG. 4 illustrates an example in which a final image 450 isgenerated in such a manner that the primary image 440 is edited by theoperator. Here, the final image 450 is generated by enlarging, by theoperator, an object portion in the primary image 440. The generatedfinal image 450 may be used as a representative image for targetcontent. For example, when the final image 450 is output with respect toa specific episode of a webtoon, the final image 450 may be displayed ata site for servicing the webtoon as a representative image for thespecific episode of the corresponding webtoon. If inspection and/oredition by the operator is unnecessary, the primary image 440 may bedisplayed at a site for servicing the webtoon as a representative imagewithout generating the final image 450.

FIGS. 5 to 8 illustrate examples of a tool for generating and managing arepresentative image according to an example embodiment. FIGS. 5 to 8illustrate screen examples of a tool for generating and managing arepresentative image (hereinafter, an image generation and managementtool 500). The image generation and management tool 500 may be producedin the form of a website and provided for an operator by the server 150.

A first box 510 indicated with dotted lines represents stages forgenerating a representative image. Here, FIG. 5 illustrates a“recognition target and pattern selection” stage. Here, an automaticstage-by-stage setting button 520 may be a user interface to which alink to a function for setting a stage for automatically processing thestages included in the first box 510 is set. For example, in response tothe operator pressing the automatic stage-by-stage setting button 520, auser interface for setting automatic processing to each of the fourstages, “recognition target and pattern selection,” “recognition resultselection,” “guide selection,” and “generated image verification,”included in the first box 510 may be provided to the operator. Withrespect to the automatic processing set stages, the server 150 mayautomatically perform corresponding processing according to acorresponding stage based on learning.

The image generation and management tool 500 may include a workselection button 531 for selecting a single webtoon from among aplurality of webtoons or all of the webtoons, an episode selectionbutton 532 for selecting an episode of the selected webtoon, and a firstpreview window 533 for previewing the selected work or episode. Also,the image generation and management tool 500 may further include a fileselection and add button 541 for receiving an input of a matchingreference image and a second preview window 542 for previewing an imageof a selected file. Here, the matching reference image may be an exampleof the reference data.

Also, the image generation and management tool 500 may include buttons551, 552, 553, and 554 for selecting a recognition pattern. Thecharacter button 551 may be a user interface for selecting the characterface recognizer 322 of FIG. 3, and the speech balloon button 552 may bea user interface for selecting the speech balloon recognizer 323 of FIG.3. Also, the text button 553 may be a user interface for selecting thetext recognizer 324 of FIG. 3, and the automation button 554 may be auser interface for setting the server 150 to automatically select apattern based on learning. In FIG. 5, the character button 551 isselected as an example.

Also, the image generation and management tool 500 may include a patternrecognition tool 560 for processing recognition of an object. In theexample embodiment of FIG. 5, in response to a selection of the operatoron the pattern recognition tool 560, processing for recognizing acharacter face in a target image of a selected webtoon “AAA” isperformed. Here, the server 150 may recognize the character face in thetarget image by using matching reference image “AAAtitle.jpg” asreference data.

If automatic processing is set for the “recognition target and patternselection” stage, the server 150 may select a work and/or episode, apattern, and reference data based on learning and may automaticallyprocess recognition of the object.

FIG. 6 illustrates a display example of the image generation andmanagement tool 500 for the “recognition result selection” stage. Here,a preview of a work or an episode that includes recognized objects asshown in boxes 611, 612, and 613 indicated with solid lines may bedisplayed on a third preview window 610. Also, previews of therecognized objects may be displayed on a fourth preview window 620.Here, the operator may directly set at least one of the objectsdisplayed on the fourth preview window 620. Alternatively, the operatormay automatically set an object for each work through an automaticwork-by-work image selection button 630 or may automatically set anobject for each episode through an automatic episode-by-episode imageselection button 640. For example, in response to a selection on awebtoon “AAA” as target content, objects may be recognized from thewebtoon “AAA.” Here, in response to a selection on the automaticepisode-by-episode image selection button 640, objects for each ofepisodes included in the webtoon “AAA” may be selected from among therecognized objects. For example, with the assumption that selectedobjects “AAA_E1_01,” “AAA_E2_01,” and “AAA_E3_01” are present for therespective episodes in the webtoon “AAA,” the server 150 mayautomatically select the object “AAA_E1_01” for the episode “AAA_E1,”the object “AAA_E2_01” for the episode “AAA_E2,” and the object“AAA_E3_01” for the episode “AAA_E3,” respectively. A recognition resultapplication button 650 may be a user interface for applying a selectedobject to a work or an episode. Here, applying an object may indicateassociating the work or the episode with an image of the selectedobject.

If automatic processing is set for the “recognition result selection”stage, the server 150 may automatically select and apply an image of anobject for a work and/or episode based on learning.

FIG. 7 illustrates a display example of the image generation andmanagement tool 500 for the “guide selection” stage. Here, previews ofimages 720 and 730 selected and applied in the “recognition resultselection” stage of FIG. 6 are displayed on a fifth preview window 710.The operator may extract a desired image using a guide for the images720 and 730 of the object. FIG. 7 illustrates an example in which squareguides 740 and 750 each with a size of 96×96 pixels are provided toupper centers of the images 720 and 730 of the object, respectively. Theoperator may select a shape (square, horizontal, vertical, etc.) of aguide, a size of a guide, and a position at which a guide recognitionarea is to be aligned through the “guide selection” stage and mayextract a desired primary image from the images 720 and 730 of theobject. For example, if the operator presses a guide application andimage generation button 760 by selecting a guide, the selected guide maybe applied, and a primary image may be extracted and generated based onthe applied guide.

Also, if automatic processing is set for the “guide selection” stage,the server 150 may generate a primary image by selecting a shape, asize, and a position of a guide based on learning.

FIG. 8 illustrates a display example of the image generation andmanagement tool 500 for the “generated image verification” stage. In the“generated image verification” stage, primary images generated in FIG. 7may be displayed. FIG. 8 illustrates an example of a reference image andselected primary images through a square guide with a size of 120×120pixels. If the operator presses an image generation button 810 afterinspection and/or edition, a final image (a representative image) may begenerated.

Although the image generation and management tool 500 provided to theoperator is described above with reference to FIGS. 5 to 8, it isprovided as an example for understanding by explaining a progressprocess of each of the stages. At least one of the “recognition targetand pattern selection” stage, the “recognition result selection” state,and the “guide selection” stage may be automatically performed by theserver 150 trained through machine learning.

For example, the server 150 may generate a representative image using animage of an object that is selected from among images of the recognizedobjects based on a degree of matching with reference data. Also, theserver 150 may generate the representative image by further usingreactions of users, for example, a user stay time which is the timeusers display each images on the screen of a user terminal, a clickthrough rate of a corresponding image, a user comment or recommendation,and the like. For example, the server 150 may measure and manage adisplay time corresponding to the user stay time for each scenedisplayed on a terminal screen of each of the users in a webtoon in aformat of a vertical image file posted to a website. The longer the userstay time measured for a plurality of users, a corresponding scene maybe determined to have a relatively high popularity and a probabilitythat an image of an object extracted in the scene may be selected mayincrease. Also, a function for posting a recommendation or a commentbased on a cut unit may be provided in a webtoon in a form of switchingbetween scenes of a cut unit. In this case, a popular cut may be setbased on a number of recommendations, a number of comments, etc., foreach cut, and an image of an object extracted from the corresponding cutmay be highly likely to be selected. As another example, a scene clickedon by a relatively large number of users may be determined as a popularscene. The server 150 may calculate a popularity for each scene or foreach cut by quantifying each of reactions of the users, and, in the caseof selecting an image of an object based on the calculated popularity,may assign a weight to an image of an object extracted from a popularscene or a popular cut.

Although a method of generating a representative image for a webtoon isdescribed with the example embodiments, it is provided as an exampleonly. Those skilled in the art may easily understand that arepresentative image may be extracted from any type of contentsincluding at least one image through the aforementioned representativeimage generation method. Types of content may include, for example, anillustration, a plurality of images related to a specific product, and aphoto album of a user.

FIG. 9 is a flowchart illustrating an example of a representative imagegeneration method according to an example embodiment. The representativeimage generation method may be performed by a computer apparatus, suchas the server 150. Here, the processor 222 of the server 150 may beconfigured to execute a control instruction according to a code of atleast one program or a code of an OS included in the memory 221. Here,the processor 222 may control the server 150 to perform operations 910to 950 included in the representative image generation method of FIG. 9in response to the control instruction provided from the code stored inthe server 150.

Referring to FIG. 9, in operation 910, the server 150 may receive targetcontent selected from among a plurality of registered contents. Forexample, the plurality of registered contents may include a plurality ofregistered webtoons, the target content may include a single webtoon ora single episode included in the single webtoon, and the generatedrepresentative image may include a representative image of the singlewebtoon or a representative image of the single episode. As describedabove, content is not limited to the webtoon and any type of contentsincluding a plurality of images or a single image in which a pluralityof scenes is connected may employ the representative image generationmethod to generate a representative image of content.

In operation 920, the server 150 may recognize an object of a presetpattern in at least one target image included in the received targetcontent. As described above, the preset pattern may include at least oneof a face pattern of a character, a speech pattern, and a text pattern.Also, depending on example embodiments, the preset pattern may bevariously set and/or learned, if necessary, such as a shape of a person,a shape of an animal, and/or a shape of a vehicle such as a car and anairplane.

The text pattern may refer to a text that represents a specific line ora specific keyword and the text may be recognized as an object. Suchtext recognition may be used for a subsequent text search or imagesearch from the content. For example, for text recognition, all of thetexts included in target images included in target content may berecognized for comparison to a specific line or a specific keyword.Here, recognized texts may be stored in association with a correspondingtarget image, and stored data may be used for the text search or theimage search from the content. In detail, an example in which a text“ratio is different” is recognized in association with a scene A of awebtoon and stored in association with the scene A may be considered. Inthis case, the text “ratio is different” may be retrieved and providedfrom content through a text search using a keyword “ratio,” or an imagefor the scene A may be retrieved and provided. Alternatively, texts andtarget images stored in association with each other may be used toprovide an image similar to an image input as a query. For example, inresponse to an input of an image related to a ratio or an imageincluding the text “ratio” as a query, the scene A may be provided as asimilar image. Also, a recognized text may be used to generate a subtextfor persons who are visually handicapped. For example, a subtext forexplaining the scene A to persons who are visually handicapped may begenerated using a text recognized from the scene A.

Also, the server 150 may use reference data to recognize an object. Forexample, in operation 920, the server 150 may receive reference data foreach preset pattern and may recognize an object in at least one targetimage based on a degree of matching with the reference data. If aplurality of objects is recognized, an object having a highest degree ofmatching with the reference data may be selected and used.

In operation 930, the server 150 may generate a recognition result imageby extracting a preset size of an area including the recognized objectin the at least one target image. Here, the area with the preset sizemay be in various shapes, for example, a polygonal shape, a circularshape, and an oval shape.

When a plurality of recognition result images is generated from at leastone target image, the server 150 may select at least one recognitionresult image from among the plurality of recognition result images basedon a user reaction to a scene or a cut of the target content. Here, theuser reaction may include at least one of a user stay time, a clickthrough rate, a number of recommendations, and a number of comments withrespect to the scene or the cut of the target content.

The user stay time may be acquired by measuring an amount of time inwhich a corresponding scene or cut is displayed on a terminal of theuser. Also, the click through rate may be acquired by measuring a numberof times the corresponding scene or cut is selected on the terminal ofthe user.

In operation 940, the server 150 may generate a representative imagerelated to the target content based on the generated recognition resultimage. For example, in operation 940, the server 150 may determine ashape, a size, and an alignment position of a guide, may apply the guidewith the determined shape and size to the generated recognition resultimage based on the alignment position, may extract a recognition areathat is recognized based on the applied guide, and may generate an imageincluding the extracted recognition area as a representative image. Theexample of using the guide is described above with reference to FIG. 7.

In operation 950, the server 150 may display the generatedrepresentative image through a site for providing the target content tousers. Although an example embodiment of generating and displaying arepresentative image for a single piece of target content is describedwith reference to FIG. 9, it may be easily understood that arepresentative image may be automatically generated and displayed withrespect to each of a plurality of target contents (for example, episodesof each of a plurality of webtoons to be uploaded on the day) byapplying the representative generation and display process to each ofthe plurality of target contents.

As described above, according to some example embodiments, it ispossible to recognize an object in content, such as a webtoon, anillustration, a plurality of images related to a specific product, aphoto album, and the like, and to automatically generate arepresentative image for the corresponding content. Also, it is possibleto provide a tool capable of automatically generating and managing arepresentative image using an image matching model.

The systems or apparatuses described herein may be implemented usinghardware components, software components, or a combination thereof. Forexample, the apparatuses and the components described herein may beimplemented using one or more general-purpose or special purposecomputers, such as, for example, a processor, a controller, anarithmetic logic unit (ALU), a digital signal processor, amicrocomputer, a field programmable gate array (FPGA), a programmablelogic unit (PLU), a microprocessor, or any other device capable ofresponding to and executing instructions in a defined manner. Theprocessing device may run an operating system (OS) and one or moresoftware applications that run on the OS. The processing device also mayaccess, store, manipulate, process, and create data in response toexecution of the software. For purpose of simplicity, the description ofa processing device is used as singular; however, one skilled in the artwill be appreciated that a processing device may include multipleprocessing elements and/or multiple types of processing elements. Forexample, a processing device may include multiple processors or aprocessor and a controller. In addition, different processingconfigurations are possible, such as parallel processors.

The software may include a computer program, a piece of code, aninstruction, or some combination thereof, for independently orcollectively instructing or configuring the processing device to operateas desired. Software and/or data may be embodied in any type of machine,component, physical equipment, virtual equipment, computer storagemedium or device, to be interpreted by the processing device or toprovide an instruction or data to the processing device. The softwarealso may be distributed over network coupled computer systems so thatthe software is stored and executed in a distributed fashion. Thesoftware and data may be stored by one or more computer readable storagemedia.

The methods according to the above-described example embodiments may beconfigured in a form of program instructions performed through variouscomputer devices and recorded in non-transitory computer-readable media.The media may also include, alone or in combination with the programinstructions, data files, data structures, and the like. The media maycontinuously store computer-executable programs or may transitorilystore the same for execution or download. Also, the media may be varioustypes of recording devices or storage devices in a form of one or aplurality of hardware components. Without being limited to mediadirectly connected to a computer system, the media may be distributedover the network. Examples of the media include magnetic media such ashard disks, floppy disks, and magnetic tapes; optical media such asCD-ROM and DVDs; magneto-optical media such as floptical disks; andhardware devices that are specially configured to store programinstructions, such as read-only memory (ROM), random access memory(RAM), flash memory, and the like. Examples of other media may includerecord media and storage media managed by Appstore that distributesapplications or a site that supplies and distributes other various typesof software, a server, and the like. Examples of program instructionsinclude both machine code, such as produced by a compiler, and filescontaining higher level code that may be executed by the computer usingan interpreter.

While this disclosure includes specific example embodiments, it will beapparent to one of ordinary skill in the art that various alterationsand modifications in form and details may be made in these exampleembodiments without departing from the spirit and scope of the claimsand their equivalents. For example, suitable results may be achieved ifthe described techniques are performed in a different order, and/or ifcomponents in a described system, architecture, device, or circuit arecombined in a different manner, and/or replaced or supplemented by othercomponents or their equivalents.

Therefore, other implementations, other example embodiments, andequivalents of the claims are to be construed as being included in theclaims.

What is claimed is:
 1. A method performed by a processor of generating arepresentative image related to content, the method comprising:recognizing an object in at least one image included in the content;generating a recognition result image by extracting an area includingthe recognized object in the at least one image; and generating therepresentative image related to the content based on the generatedrecognition result image.
 2. The method of claim 1, wherein therecognizing of the object comprises recognizing an object correspondingto at least one pattern in the at least one image, and the at least onepattern comprises at least one of a facial pattern of a character, aspeech balloon pattern, and a text pattern.
 3. The method of claim 1,wherein the recognizing of the object comprises: receiving referencedata for each of at least one pattern; and recognizing the object in theat least one image based on matching with the reference data.
 4. Themethod of claim 1, wherein the generating of the representative imagecomprises: determining a shape, a size, and an arrangement position of aguide; applying the guide with the determined shape and size to thegenerated recognition result image based on the arrangement position andextracting a recognition area that is recognized based on the appliedguide; and generating an image including the extracted recognition areaas the representative image.
 5. The method of claim 1, furthercomprising: in response to a plurality of recognition result imagesbeing generated from the at least one image, selecting at least onerecognition result image from among the plurality of recognition resultimages based on a user reaction to a scene or a cut of the content. 6.The method of claim 5, wherein the user reaction comprises at least oneof a user stay time, a click through rate, a number of recommendations,and a number of comments with respect to the scene or the cut.
 7. Themethod of claim 1, further comprising: regenerating the representativeimage based on a change in a user reaction to a scene or a cut of thecontent.
 8. The method of claim 7, wherein the user reaction comprisesat least one of a user stay time, a click through rate, a number ofrecommendations, and a number of comments with respect to the scene orthe cut.
 9. The method of claim 1, wherein the content comprisesregistered webtoon content or at least one episode included in thewebtoon content, and the generated representative image comprises arepresentative image of the webtoon content or a representative image ofthe episode.
 10. The method of claim 1, further comprising: displayingthe generated representative image through a site for providing thecontent to users.
 11. A non-transitory computer-readable record mediumstoring a program which, when executed by a computer, performing themethod of generating a representative image according to claim
 1. 12. Acomputer apparatus comprising: at least one processor configured toexecute a computer-readable instruction, wherein the at least oneprocessor is configured to recognize an object in at least one imageincluded in content, generate a recognition result image by extractingan area including the recognized object in the at least one image, andgenerate a representative image related to the content based on thegenerated recognition result image.
 13. The computer apparatus of claim12, wherein the recognizing of the object comprises recognizing anobject corresponding to at least one pattern in the at least one image,and the at least one pattern comprises at least one of a facial patternof a character, a speech balloon pattern, and a text pattern.
 14. Thecomputer apparatus of claim 12, wherein the recognizing of the objectcomprises: receiving reference data for each of at least one pattern,and recognizing the object in the at least one image based on matchingwith the reference data.
 15. The computer apparatus of claim 12, whereinthe generating of the representative image comprises: determine a shape,a size, and an arrangement position of a guide, apply the guide with thedetermined shape and size to the generated recognition result imagebased on the arrangement position and extract a recognition area that isrecognized based on the applied guide, and generate an image includingthe extracted recognition area as the representative image.
 16. Thecomputer apparatus of claim 12, wherein, in response to a plurality ofrecognition result images being generated from the at least one image,the at least one processor is configured to select at least onerecognition result image from among the plurality of recognition resultimages based on a user reaction to a scene or a cut of the content. 17.The computer apparatus of claim 12, wherein the at least one processoris configured to regenerate the representative image based on a changein a user reaction to a scene or a cut of the content.